Indexing

Learn how the indexing works in the Algolia extension for Magento Open Source and Adobe Commerce.

To provide fast and relevant search, the Algolia extension for Magento Open Source and Adobe Commerce takes your products, categories, and pages from your store and transforms them into searchable data. This process is called indexing.

You can index your data through these methods:

• Automatically with an indexing queue
• Manually from the Magento console
• Manually on the command-line

• Algolia extension for Magento Open Source and Adobe Commerce

• Send and update your data

Check the troubleshooting guide if you have any issues with your data, indexes, or queue.

To provide the best search experience for your users, the extension automatically keeps all your data up to date, offering two indexing mechanisms in Magento:

• Section reindex. An entire section of the catalog (products, categories, pages) is uploaded to Algolia’s servers and reindexed.
• Single item reindex. - A single resource (product, category, page) is uploaded to the servers and reindexed. Single item reindexing happens when a resource is updated.

By default, the indexing operations run synchronously, and you must wait until the indexing process finishes before you can continue. This is inconvenient and can have unexpected side effects. Therefore, you should instead use the indexing queue, which processes all index operations in the background and avoids some sources of errors.

Section reindex with an indexing queue

The indexing queue uses temporary indices to reindex products. Instead of updating the production index directly, Algolia creates a copy and replaces the production index only when the update is complete.

This approach has several advantages:

• Higher reindexing speed
• No inaccuracies with deleted products
• Fewer indexing operations

Changes to the index will be visible once the temporary index completely replaces the production index.

Section reindex without an indexing queue

Without the indexing queue, reindexing the complete catalog is synchronous: everything has to wait until the process completes.

Processing large indices synchronously may trigger PHP timeouts.

Without a queue, reindexing takes more time and resources. It’s also less reliable: if products are updated while the reindexing is in process, the data on Algolia’s servers may not be up to date.

Enable the indexing queue for any full reindexing, especially if you have large catalogs.

Automatic indexing

The Algolia extension for Magento sends every change and deletion on products or categories to Algolia’s servers to keep all data up to date.

Manual indexing

To index your data manually, first turn off the indexing queue: set Stores > Configuration > Algolia Search > Indexing Queue/Cron > Enable Indexing Queue to No in the Magento dashboard.

You can now use the command-line to send manual updates to Algolia’s server. For example, to completely reindex all products:

sh

php path/to/magento/bin/magento indexer:reindex algolia_products

Use the same command for all other indices created by the extension:

Command	Action
algolia_products	Reindex all products
algolia_categories	Reindex all categories
algolia_pages	Reindex all CMS pages
algolia_suggestions	Reindex all search query suggestions
algolia_additional_sections	Reindex all additional sections
algolia_queue_runner	Process jobs in the indexing queue
algolia_delete_products	Remove inactive products from Algolia indices

Manual indexing with the indexing queue turned on

To reindex with the indexing queue turned on, manually trigger the queue runner, or let the cron job process it when your data is updated. To do this, and add all your products to the indexing queue, run:

1. php path/to/magento/bin/magento indexer:reindex algolia_products
2. PROCESS_FULL_QUEUE=1 php path/to/magento/bin/magento indexer:reindex algolia_queue_runner

You should only implement this as a troubleshooting activity, not as a regular task.

Indexing products

It’s essential for your business to have exact and up-to-date product data in your search.

Full reindexing

To fully reindex all your products:

sh

php path/to/magento/bin/magento indexer:reindex algolia_products

Indexable products

To prevent too many indexed products and save indexing operations, Algolia only indexes products that actually appear in the online store.

Algolia only indexes products that are:

• Visible - either in the catalog, the search, or both
• Enabled
• Not deleted
• In stock - unless Magento is configured to show out-of-stock products, too.

If you miss a product in your search, make sure the product meets all four criteria.

• Troubleshooting Data, Index, and Queue Issues

Missing products

If search can’t find some products, go the Algolia dashboard, select the index that contains your products and search there.

• If you can find the product, confirm that is has the correct visibility setting. If products are only visible in the catalog, they won’t show up while searching. They will only show up on the InstantSearch page (in the category section).
• If you can’t find the product, it hasn’t been indexed. Make sure it adheres to the indexing requirements. If the product meets the requirements, Reindex it.

If the problem persists, turn on logging and investigate the log files to see what’s going on during the reindexing process.

Searchable attributes

You can configure which attributes should be searched when users type their queries. To configure the list of searchable attributes, go to Stores > Configuration > Algolia Search > Products.

In the Products tab, you can set each attribute as searchable, retrievable, or ordered. By default, all attributes are set to be searched as unordered. This is usually the best choice, and you shouldn’t change it without specific reasons.

• Configuring searchable attributes the right way

Default indexed attributes

Some attributes are always indexed, regardless of your configuration. These attributes aren’t all searchable but can be used for filtering, sorting, customizing the ranking, and building the results page.

These product attributes are always indexed:

Attribute	Description
name	Name
url	URL
visibility_search	Visibility in the search
visibility_catalog	Visibility in the catalog
categories	Categories, formatted as a tree path
categories_without_path	Categories, without the tree path
thumbnail_url	Thumbnail image
image_url	Main image
in_stock	Stock availability
price	Price
type_id	Type (simple, configurable, bundled, etc.)

image_url start with /pub

This error can occur when your app has the pub/ directory as the root directory for the Magento installation. This is usually considered a good practice, but it has some drawbacks. The extension can’t know which directory is the root folder for Magento, so it always assume the base directory is your root folder.

The best way to handle this problem is to make your server ignore /pub in the URL.

If it’s not possible to change the server configuration, a setting is provided in Stores > Configuration > Algolia Search > Advanced > Remove /pub/ from image URLs. When this setting is changed, reindex the catalog.

Facets

Facets are attributes used to filter search results. Common facets include price, color, categories, and brand. You may need to adjust the facets of the products you sell and how your users search for these products.

For each facet, you can specify:

• The attribute
• The label - this will be displayed above the filter
• The facet type

Select one of the modifiers from the Options drop-down menu to make facet values searchable, not searchable, or filterable (not “facetable”).

Use the filterOnly modifier if you only want to use the filtering feature. This will help optimize search speed and reduce index size.

You can attach facets to rules. If you set Create Query rule to Yes, Algolia uses the attribute as a filter. Suppose you create a rule for the color attribute and users search for products with a specific color, for example, “red shorts”. In that case, Algolia filters all results with “red” as the color attribute.

When a rule is attached to an attribute, it’s applied to both the Autocomplete menu and InstantSearch results page.

Rules are only available on Grow and Premium plans.

By default, the extension provides facets on the price, categories, and color attributes. Any numeric attribute, for example, price, is shown as a slider in the user interface.

Attributes specified as facets are automatically indexed as retrievable but not searchable. You don’t need to add them manually to the searchable attributes.

How to add numeric attributes as non-castable

To access the configuration page for Algolia in your Magento project, go to Stores > Algolia Search > Credentials and Basic Setup > Advanced in the Admin panel.

From there, you can specify which text fields or non-castable attributes to include under Product numeric textfield attributes that should not be converted into a numeric field by Magento.

‘No’ values in filters

If a product doesn’t have any value assigned to an attribute, the value of this attribute will be automatically set to ‘No’ by Magento. To turn off indexing for ‘No’ values, set the Index empty value setting in your searchable attributes to No for the attributes you want to ignore ‘No’ values for.

Changing any index setting requires a full reindex of all data.

Sorting strategies

Sorting is only available on the InstantSearch Results page. By default, the Algolia Magento 2 extension uses the exhaustive sorting strategy with Standard Replicas, but you can switch to the relevant sorting strategy using Virtual Replicas.

When searching for products, users may expect multiple ways to sort the result set. For example, they want to sort by relevance, popularity, price, or date.

The default sorting strategy when searching is sorting by relevance. Any other sorting strategy needs to be defined in the Sort Settings.

For each strategy, define an attribute, sort order (ascending or descending), and label.

You can also specify whether to create a virtual replica (instead of a standard replica) for the sorting strategy.

By default, there are three sorting strategies:

1. From lowest price to highest price
2. From highest price to lowest price
3. From newest to oldest

If you’ve chosen to use standard replicas, each sorting strategy creates a new standard replica index, which increases the number of records used by your plan.

Attributes configured for sorting are automatically indexed as retrievable but not searchable. You don’t need to add them manually to the searchable attributes.

Replicas are configured in Algolia whenever you save changes to the sorting configuration in your store’s Admin. If a sorting attribute is deleted from the configuration in your store’s Admin, the corresponding replica index will be deleted in Algolia.

If you experience issues with your replica configuration in Algolia or can’t save changes from your store’s Admin, refer to the Troubleshooting guidance.

You can now also apply sorting strategies to your InstantSearch results pages in your store’s Admin with Merchandising Studio. With this method, you can add rules that apply alternative sorting strategies when certain conditions are met. The extension doesn’t manage these indices.

Number of records created

To provide fast results, Algolia pre-computes a part of the order at indexing time. This means only one sorting is available for a single index. Each additional sort will result in an additional index. Sorting indices may take up significantly increased storage depending on whether standard (default) or virtual replicas are enabled for the corresponding sorting attribute configuration.

In Magento, this means that by default these indices are created:

• One per store
• One per store per additional sort order (such as by price or by date)

To calculate the amount of records this generates, take a look at the following example:

A Magento app has two stores (for two languages) which both have 100 products. Both stores have additional sorts on price and date, resulting in a total of four additional sorts.

Using this scenario, you would need to index:

Sort	Records	Total
Default sorting	100×2	200
Custom sorting	100×2×4	800
Total records		1000

If the customer group feature is enabled, each customer group will result in an extra index on the pricing sorting. Following the preceding example, but with a total of 5 customer groups will result in:

Sort	Records	Total
Default sorting	100×2	200
Pricing sorting	100×2×2×5	2000
Custom sorting	100×2×2	400
Total records		2600

The number of indexed records can be lowered by either removing some sorting strategies or by converting them to using virtual replicas. (Virtual replicas do require some additional storage but this is marginal relative to the number of records.)

Sorts can be configured in the extensions’ administration panel:

Switching to virtual replicas

Virtual replicas are not available on the Grow plan.

To configure Virtual Replicas for Sorting, go to Stores > Configuration > Algolia Search > InstantSearch Results Page > Sorting.

In the grid, enable virtual replicas for each desired attribute by setting Enable Virtual Replica? to Yes.

When you switch a sorting attribute to a virtual replica, Algolia deletes the standard replica and creates a virtual replica index in its place. If you reset this value to No, the opposite will happen.

Virtual replicas slightly increase the primary index’s size (less than 10 MB per million records), but they don’t affect the number of records. For more information, see Standard and virtual replicas.

You can enable up to 20 virtual replicas per index. The extension will inform you if you have exceeded this limit.

Scoped configuration

The extension will create a product index for each Algolia-enabled store (or “store view”) in your Magento instance. To keep within the virtual replica limits on a single index, use the store view selector to configure your sorting attributes for each store independently.

Choose the website or store view where you wish to apply the desired changes.

For a website scope, remove the Use Default check mark and configure your sorting attributes accordingly:

For a store scope, remove the Use Website check mark:

For more information, see Adobe’s official documentation.

Index settings

Configure these settings in the Magento dashboard:

• searchableAttributes
• customRanking
• unretrievableAttributes
• attributesForFaceting
• maxValuesPerFacet
• removeWordsIfNoResults

Manage additional index settings in the Algolia dashboard. To change the settings programmatically, you can hook into the algolia_products_index_before_set_settings event provided by the extension. See Custom backend events for more information.

Any changes in the Algolia dashboard override these settings until you perform a full reindex from the Magento dashboard.

Indexing categories

To keep the number of records and indexing operations as low as possible, Algolia only indexes active categories. To show all categories in the Autocomplete menu and InstantSearch Results page, set Show categories that are not included in the navigation menu to Yes.

Full reindexing

sh

php path/to/magento/bin/magento indexer:reindex algolia_categories

Searchable attributes

You can configure which attributes should be searched when users type their queries. To configure the list of searchable attributes, go to Stores > Algolia Search > Categories.

In the Categories tab, you can set each attribute as searchable, retrievable, or ordered. By default, all attributes are set to be searched as unordered. This is usually the best choice, and you shouldn’t change it without specific reasons.

• Configuring searchable attributes the right way

Default searchable category attributes

Some attributes are always indexed, regardless of your configuration. These attributes aren’t all searchable but can be used for filtering, sorting, customizing the ranking, and building the results page.

These category attributes are always indexed:

Attribute	Description
name	Name
url	URL
path	Path (parent categories)
level	Level in the category tree
include_in_menu	Visibility in the menu
_tags	Filled automatically by the extension
popularity	Popularity
product_count	Amount of products

Index settings

Configure these settings in the Magento dashboard:

• searchableAttributes
• customRanking
• unretrievableAttributes

Manage additional index settings in the Algolia dashboard. To change the settings programmatically, you can hook into the algolia_categories_index_before_set_settings event provided by the extension. See Custom backend events for more information.

Any changes in the Algolia dashboard override these settings until you perform a full reindex from the Magento Dashboard.

Indexing pages

To turn indexing of CMS pages on or off, go to Stores > Algolia Search > Credentials and Basic Setup > Enable Pages Index. When set to Yes the Algolia extension automatically indexes CMS pages, allowing users to search for pages in the Autocomplete menu. All active pages are indexed by default.

You can exclude specific pages, like error pages, so they don’t appear in the search results.

Turn off this indexing of pages in Additional Sections.

Full reindexing

sh

php path/to/magento/bin/magento indexer:reindex algolia_pages

Searchable attributes

You can’t configure the searchable attributes for pages in the Magento dashboard. However, you can change them programmatically, by hooking into the algolia_after_create_page_object event provided by the extension. See Custom backend events for more information.

Default searchable page attributes

These attributes are indexed by default and aren’t all searchable (some are). You can use them for filtering, sorting, custom ranking and building the search results page.

These page attributes are always indexed:

Attribute	Description
name	Name
url	URL
slug	Slug
content	Content

Any records larger that your plan’s limit won’t be indexed. In this case, only the page’s name would be searchable.

Index settings

The following settings are always set and can’t be changed through the extension’s interface:

• searchableAttributes: unordered(slug), unordered(name) and unordered(content)
• attributesToSnippet: content:7

Manage additional index settings in the Algolia dashboard. You can also change the settings programmatically, by hooking into the algolia_pages_index_before_set_settings event provided by the extension. See Custom Backend Events for a list of events provided by the extension.

Any changes in the Algolia dashboard override these settings until a full reindex is performed from the Magento Dashboard.

Indexing suggestions

Magento stores every query, the number of results, and the number of searches in the catalogsearch_query table in the database.

Magento only stores backend searches. As-you-type searches aren’t stored.

Algolia’s Magento extension lets you index queries that are performed regularly. In the settings, you can filter relevant queries, for example, by the minimum number of results or minimum popularity.

To turn this behavior on or off, go to Stores > Algolia Search > Credentials and Basic Setup > Enable Query Suggestions Index

To ensure the data in the suggestion index is relevant, enable backend search with the Algolia extension by turning on Enable Search and Make SEO Request in the Magento dashboard.

With these options enabled, the extension processes backend searches. Since the data in catalogsearch_query is also updated, the queries in this table will become more relevant over time.

By default, suggestions aren’t indexed. When enabling the indexing of suggestions, you need to trigger a manual reindex. Another way to start the indexing of suggestions is by adding a recurring job to the cron table:

txt

1 * * * * php path/to/magento/bin/magento indexer:reindex algolia_suggestions

Instead of Magento suggestions, you can also use Algolia’s Query Suggestions feature.

• Implement Algolia’s Query Suggestions

Full reindexing

sh

php path/to/magento/bin/magento indexer:reindex algolia_suggestions

Searchable attributes

You can’t configure searchable attributes for suggestions in the Magento dashboard. However, you can change them programmatically by hooking them into the algolia_after_create_suggestion_object event provided by the extension.

See Custom backend events for more information.

Default searchable query attributes

These attributes are always indexed and aren’t all searchable. You can use these attributes for filtering, sorting, custom ranking, and building the search results page.

These query attributes are always indexed:

Attribute	Description
query	Value
number_of_results	Number of results
popularity	Number of searches
updated_at	Last update timestamp

Index settings

The following settings are always set and can’t be changed in the Magento dashboard:

• searchableAttributes: unordered(query)
• customRanking: desc(popularity), desc(number_of_results)
• typoTolerance: false
• attributesToRetrieve: query
• removeWordsIfNoResults: lastWords

Manage additional settings in the Algolia dashboard. To change the settings programmatically, you can hook into the algolia_suggestions_index_before_set_settings event provided by the extension. See Custom backend events for more information.

Any changes in the Algolia dashboard override these settings until you perform a full reindex from the Magento dashboard.

Indexing additional sections

The Autocomplete menu lets you display other sections from attributes, such as colors and brands.

For this feature to work, you must enable the InstantSearch Results Page.

The attributes used for the additional sections have to be set as attributes for faceting.

sh

php path/to/magento/bin/magento indexer:reindex algolia_additional_sections

Searchable attributes

Manage additional index settings in the Algolia dashboard. It’s also possible to change the settings programmatically, by hooking into the algolia_additional_sections_index_before_set_settings event provided by the extension.

See Custom Backend Events for a list of events provided by the extension.

Default searchable attributes

These attributes are indexed by default and aren’t all searchable (some are). They can be used for filtering, sorting, customizing the ranking and building the results page.

The attributes that are always indexed:

Attribute	Description
	value	The attributes’ value, for example, Red, XL, Nike.

Index settings

The following settings are always set and can’t be changed through the extension’s interface:

• searchableAttributes: unordered(value)

Manage additional index settings in the Algolia dashboard. It’s also possible to change the settings programmatically, by hooking into the algolia_additional_sections_index_before_set_settings event provided by the extension. See Custom Backend Events for a list of events provided by the extension.

Any changes in the Algolia dashboard override these settings until a full reindex is performed from the Magento Dashboard.

Removing inactive products

To remove products from your Algolia index, use the algolia_delete_products indexer:

sh

php path/to/magento/bin/magento indexer:reindex algolia_delete_products

This indexer removes all products from Algolia indices that you don’t want to include in your search. This can be useful when you delete or deactivate products directly in Magento’s database, and the extension could reindex and remove them regularly.

The reindexer doesn’t delete products from your Magento database. Products are only removed from Algolia indices.

Incorrect links

Sometimes, links of the Magento installation can show up with an additional /magento/ in the URL. Most likely, this is because web server rewrites are turned off. Go to Stores > Configuration > General > Web > Search Engine Optimization and set the field Use Web Server Rewrites to Yes. When this setting is updated, a full reindex is required for the changes to take effect.